ANALYTICS ENGINEERING / SERVERLESS ELT

Daily market intelligence.
Zero servers to babysit.

A Medallion ELT pipeline that ingests Vietnam gold prices daily on GitHub Actions, models Bronze–Silver–Gold layers with dbt on BigQuery, and publishes trends, brand spreads and day-over-day volatility to a self-updating README dashboard — with Discord alerts when things move.

View source code ↗

ROLE

Analytics Engineer

SCOPE

Individual project

DATA

VN gold prices · 8 brands tracked daily

STACK

Python · GitHub Actions · dbt · GCS + BigQuery · Discord

01 / CONTEXT & PROBLEM

Prices are published daily. History and comparison are not.

Vietnamese gold prices vary by brand, and the questions people actually ask — how has the spread between brands moved, how volatile was this week, is today's jump unusual — need accumulated history and modeled comparisons. The constraint I set: the whole loop must run unattended and cost nothing, using GitHub Actions as both the scheduler and the compute.

02 / ARCHITECTURE

Medallion layers on a cron budget

Price sources

8 VN gold brands

→

Python ingestion

daily GitHub Actions schedule

→

Bronze on GCS

raw immutable snapshots

→

dbt on BigQuery: Silver → Gold

cleaned history → trends · spreads · volatility

→

README dashboard + Discord

self-updating charts · movement alerts

03 / KEY DECISIONS & RESULTS

Constraints as design fuel

GitHub Actions as the entire runtime

No VM, no orchestrator to patch: the scheduler, compute and deployment are one workflow file. The trade-off — no long-running processes — fits a once-daily batch perfectly.

README as the dashboard surface

The repo page is where visitors already land, so the pipeline regenerates its charts there on every run — the project demos itself with zero hosting.

Immutable Bronze snapshots

Every day's raw payload lands untouched in GCS before any transformation — modeling bugs are re-runs against preserved history, never data loss.

100%

unattended daily loop: ingest → model → publish → alert

8

gold brands tracked with spread & volatility comparisons

$0

infrastructure cost — free tiers end to end

04 / WHAT I'D DO DIFFERENTLY

Harden the source boundary.

Public price pages change without notice; the pipeline should detect schema drift and alert on missing brands rather than silently thinning the history. Source freshness tests and a dead-letter path for malformed snapshots are the next additions.